Search Results

The Programme for International Student Assessment (PISA) surveys take place every three years to measure how well 15-year-old students can apply the knowledge and skills they have learned at school to meet the challenges of today's knowledge societies. PISA 2009 is the fourth PISA survey and includes a computer-delivered assessment of digital reading. This report is concerned with the technical aspects used to create the PISA 2009 database which contains information on 470,000 students from 65 countries. The chapters are: Programme for International Student Assessment: an overview; Test design and test development; The development of the PISA context questionnaires; Sample design; Translation and verification of the test and survey material; Field operations; Quality assurance; Survey weighting and the calculation of sample variance; Scaling PISA cognitive data; Data management procedures; Sampling outcomes; Scaling outcomes; Coding reliability studies; Data adjudication; Proficiency scale construction; Scaling procedures and construct validation context; Digital reading assessment; International database. Show less

In the context of existing 'quantitative'/'qualitative' schisms, this paper briefly reminds readers of the current practice of testing for statistical significance in social science research. This practice is based on a widespread confusion between two conditional probabilities. A worked example and other elements of logical argument demonstrate the flaw in statistical testing as currently conducted, even when strict protocols are met. Assessment of significance cannot be standardised and requires knowledge of an underlying figure that the analyst does not generally have and cannot usually know. Therefore, even if all assumptions are met, the practice of statistical testing in isolation is futile. The question many people then ask in consequence is - what should we do instead? This is, perhaps, the wrong question. Rather, the question could be - why should we expect to treat randomly sampled figures differently from any other kinds of numbers, or any other forms of evidence? What we could do 'instead' is use figures in the same way as we would most other data, with care and judgement. If all such evidence is equal, the implications for research synthesis and the way we generate new knowledge are considerable.

Published abstract reprinted by permission of the copyright owner. Show less

Notwithstanding an impressive research tradition on key skills, no clear statistical criterion exists that is suitable to determine which skills may be considered key skills. This contribution proposes one possible methodology that can be used to identify key skills. Proposing an economic definition of the key skill concept and disentangling the direct and indirect effects of skills on productivity, we develop an empirical criterion for the identification of key skills. We apply this methodology to a dataset of employed vocational education graduates. We find that problem-solving skills, independence, oral presentation/speaking skills, accuracy/carefulness and initiative/creativity may be considered key skills.

Published abstract reprinted by permission of the copyright owner. Show less

The prevailing discourse informing most Canadian training and labour market policy assumes a positive link between individuals’ training and their labour market returns in the new knowledge economy. The primary objective of the study is to test the current rhetoric by developing a statistical model of women’s job-related training. Training participation is a complex and multi-dimensional social phenomenon. Within the arsenal of existing statistical methods, structural equation modeling is one of the few methods with a capacity to represent complex phenomenon by simultaneously testing cause-and-effect hypotheses. The study uses structural equation modelling to develop, test, and evaluate a model of the determinants and rewards of women’s job related training. Empirical studies of job-related training in Canada are few. Until the recent launch of a linked, employer-employee survey, there had been a paucity of data sources that facilitate a national level analysis of job-related training. By using this new data source, the study makes a significant contribution filling the existing gap in our understanding of the determinants of, and returns to women’s training in Canada. The study confirms that women are deriving significant economic benefits from their training participation. The conclusion drawn from the national-level patterns in Canada is that training is a crucial element in the reward structure of the labour market for women, as it plays a dual role of being both a reward in itself and a predictor of other labour market rewards.

Published abstract reprinted by permission of the copyright owner. Show less

The history of learning environment research is characterised by the creation and application of rating scale instruments to elicit attitudinal data from students and teachers about the learning environment. For several decades, the data from these instruments have been subject to various types of statistical analysis. Typically, such analyses are applied as part of the instrument development process as well as to inform answering of questions about the attributes of learning environments, the influences on learning environments, and the temporal stability of learning environments. Notwithstanding the widespread use of statistical techniques in quantitative learning environment research, these techniques do not necessarily reflect the knowledge and methods developed in the fields of objective measurement and rating scale analysis. It is therefore timely and appropriate to examine the implications of objective measurement theory and practice for the use of rating scales in learning environment research. The objective of this paper is to apply a measurement perspective to examine the issues involved in the development of ratings scales and in the analysis of the data they elicit. An argument is made that using raw scores from rating scale instruments for subsequent arithmetic operations and applying linear statistics is less preferable than using measures.

Published abstract reprinted by permission of the copyright owner.

Individual papers from the conference are indexed from TD/TNC 94.97 to TD/TNC 94.100 and from TD/TNC 94.301 to TD/TNC 94.319. Show less

This technical paper provides an evaluation of three alternative methodological approaches for calculating completion rate projections for apprentices and trainees. Projected rates are calculated for recently commencing cohorts because many contracts are still active, and the actual completion rate is not yet known. Completion rate projections were calculated for cohorts commencing in 2013, 2014, and 2015, and evaluated against actual completion rates for these cohorts.

All three methods generated projections within five percentage points of actual completion rates. However, the work revealed differences in the consistency of error rates, data requirements, and the timeliness with which projections could be generated for new cohorts.

Published abstract. Show less

This report describes, analyses and compares information collected about training costs and participation in training from organisations such as national statistics offices (NSOs) and national vocational training institutions (NVTIs) in Asia and the Pacific, Africa and Latin America and the Caribbean. This analysis and comparison was intended to identify weaknesses and strengths in the availability of and access to statistics on investment in training activities. The document concludes by summarising the major findings, including a determination that a policy context that generates demand for indicators on costs and participation in training as a central instrument for policy assessment and monitoring is critical to the production and collection of effective data. Show less

There has been much talk recently about looming shortages in Canada's supply of individuals working in health occupations. To understand the degree to which the supply of workers in health occupations is meeting (and will meet) the health needs of Canadians, Health Canada - through the Advisory Committee on Health Delivery and Human Resources (ACHDHR) - contracted the Centre for Education Statistics at Statistics Canada to study the relationship between education and training and the supply of entrants into health occupations. Using major Statistics Canada data sources related to the education and training of Canadians, the intent of this second data report from the Health Human Resources and Education Project is to provide a jurisdictional view of the information already released at the national level, as well as to provide information (at the Canada and jurisdictional levels) on additional topics that were identified in the information needs outline that were not covered in the first data report, which was released in 2007. As such, this report reveals some important information about what happens before, during and after health education. It focuses on interest in health occupations, the number of students taking and graduating from postsecondary health programs along with their socio-demographic characteristics and those of the faculty teaching these programs, the labour market experiences of recent graduates from these programs - including their mobility after graduation - as well as the ongoing participation of health workers in formal and informal training.

Excerpt from publication. Show less

A National Skills Task Force was established to assist the United Kingdom (UK) government in developing a National Skills Agenda. The Task Force provided evidence on the nature, extent and pattern of skill needs and shortages in the UK and possible directions for their future development. This evidence was summarised in 'Skills for all: research report from the National Skills Task Force', published in June 2000 [available in VOCEDplus at TD/TNC 62.657]. This report is one of a number of associated reports containing detailed information on various aspects of this research. These reports are indexed from TD/TNC 65.225 to TD/TNC 65.232 and at TD/TNC 71.314. Other reports related to the National Skills Task Force are indexed from TD/TNC 62.653 to TD/TNC 62.656 and TD/TNC 62.658.

This technical paper addresses four questions: (1) what surveys of skill deficiencies are currently produced in the UK? (2) how do respondents answer the questions in the surveys? (3) what can we learn from basic written reports and analysis of the survey results? and (4) do more sophisticated analytical methods alter views of the value of the survey information? The final section of the paper presents some brief conclusions about how useful survey information is for the identification of skill deficiencies. The paper includes a glossary of technical terms. An annex outlines current survey questions from the three main surveys currently in use in the UK. Show less

The 2003 National Assessment of Adult Literacy (NAAL) assessed the English literacy skills of a nationally representative sample of 18,500 U.S. adults (age 16 and older) residing in private households. NAAL is the first national assessment of adult literacy since the 1992 National Adult Literacy Survey (NALS). The NAAL and NALS produced direct estimates of Prose, Document, and Quantitative literacy, each reported on a 0 to 500 scale and on four performance levels: Below Basic, Basic, Intermediate, and Proficient based on this scale. This report describes the statistical methodology used to produce the model-dependent - indirect - estimates of the percentages of adults at the lowest literacy level for individual states and counties for 1992 and 2003. The county and state indirect estimates themselves are provided at the NAAL website (the state indirect estimates are also provided in appendices to this report). The measure chosen for the indirect estimation is the percentage of adults lacking Basic prose literacy skills (BPLS). The literacy of adults who lack BPLS ranges from being unable to read and understand any written information in English to being able to locate easily identifiable information in short, commonplace prose text, but nothing more advanced. It should be noted that adults who were not able to take the assessment because they were not able to communicate in English or Spanish (i.e. language barrier cases) are included in the indirect estimates and classified as lacking BPLS because they can be considered to be at the lowest level of English literacy.

Published summary reprinted by permission of the copyright owner. Show less